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Connected Mathematics Project (CMP) is a math curriculum for 
students in grades 6-8. It uses interactive problems and everyday 
situations to explore mathematical ideas, with a goal of fostering 
a problem-centered, inquiry-based learning environment. At each 
grade level, the curriculum covers numbers, algebra, geometry/mea- 
surement, probability, and statistics. 

Research 2 

The What Works Clearinghouse (WWC) identified two studies of CMP 
that both fall within the scope of the Primary Mathematics topic area 
and meet WWC group design standards. 3 No studies meet WWC 
group design standards without reservations; the two studies meet 
WWC group design standards with reservations. Together, these 
studies included 3,062 students in grades 6-8 in at least 23 schools 
in 10 locations. 4 

The WWC considers the extent of evidence for CMP on the mathe- 
matics achievement of students in primary mathematics courses to be 
medium to large for the mathematics achievement domain, the only 
domain examined for studies reviewed under the Primary Mathemat- 
ics topic area. 5 (See the Effectiveness Summary on p. 4 for more details 

Effectiveness 

CMP was found to have no discernible effects on mathematics achievement for students in primary 
mathematics courses. 


Table 1. Summary of findings 6 
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(percentile points) 
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effectiveness 
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Extent of 
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Mathematics achievement 

No discernible effects 

+2 

0 to +4 

2 

3,062 

Medium to large 
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Intervention Information 

Background 

CMP was developed by Michigan State University and is distributed by Pearson Education. Address: P.O. Box 
6820, Chandler, AZ 85246. Email: k12customerservice@pearson.com. Web: https://connectedmath.msu.edu/. 
Telephone: (800) 848-9500. 

Intervention details 

CMP is an inquiry-based mathematics curriculum. Mathematical ideas are embedded in sequenced sets of tasks 
and explored in depth to help students develop a thorough understanding of mathematical concepts. Throughout 
the curriculum, students focus on problem-solving strategies, communicating their reasoning, offering proofs, and 
using representations. 

CMP includes four courses: Grade 6 Mathematics, Grade 7 Mathematics, Grade 8 Mathematics, and Algebra I. 
Each course is organized into units, with seven to eight units per course, with units organized around a mathemati- 
cal concept or set of related concepts, such as area and perimeter, or operations on fractions. 7 

CMP seeks to promote students’ mathematical knowledge and understanding by helping them understand the 
connections among mathematics topic areas and between mathematics and other academic subjects. Students 
move flexibly between graphic, numeric, symbolic, and verbal representations to develop fluency in conceptual and 
procedural knowledge. 

The publisher is currently selling the third edition of CMP. The developer’s website describes how the curriculum 
has been refined and enhanced from the first and second editions. 8 


Cost 

As of July 201 6, the cost of CMP varies between $73.97 and $1 27.97 per student for the full curriculum, depending 
on the grade level and type of implementation (digital, print, or a combination of formats). Manipulatives kits cost 
between $221 .97 and $313.47 each, and a teacher resource kit costs $524.97. More detailed cost information is 
available from the publisher. 
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Research Summary 

The WWC identified 24 eligible studies that investigated the effects of 
CMP on the mathematics achievement of primary students. An addi- 
tional 76 studies were identified but do not meet WWC eligibility criteria 
for review in this topic area. Citations for all 100 studies are in the Ref- 
erences section, which begins on p. 5. 

The WWC reviewed 24 eligible studies against group design standards. 

None of the 24 studies are randomized controlled trials that meet WWC group design standards without reservations. 
Two of the 24 studies use quasi-experimental designs and meet WWC group design standards with reservations. Those 
two studies are summarized in this report. The remaining 22 studies do not meet WWC group design standards. 

Summary of studies meeting WWC group design standards without reservations 

No studies of CMP met WWC group design standards without reservations. 

Summary of studies meeting WWC group design standards with reservations 

Cai, Wang, Moyer, Wang, and Nie (2011) examined the effect of the CMP curriculum by matching seven schools 
using the CMP curriculum to seven schools not using the curriculum on comparable demographic characteristics. 
All 14 schools were located in a single large, urban school district. Students in the intervention schools used CMP 
in the sixth, seventh, and eighth grades. Students in the comparison schools did not use CMP. The study involved 
a single cohort of students who were followed over six school years (2005-06 through 201 0-1 1), from grade 6 to 
grade II. 9 All but the analyses of ninth and tenth grade outcomes meet WWC group design standards with reserva- 
tions. The WWC based its effectiveness rating on outcomes measured at the end of eighth grade. The eighth-grade 
analytic sample included 303 students in the CMP group and 303 students in the comparison group in 14 schools. 
The study used the first edition of CMP. 

Ridgway, Zawojewski, Hoover, and Lambdin (2002) conducted a study using a quasi-experimental design in which 
two classrooms from each of nine sites across the country used CMP. Five sites were in the Midwest, two were in the 
West, and two were in the East. For every two CMP classrooms, one non-CMP classroom was recruited for the study. 
The authors matched the CMP classrooms to non-CMP classrooms based on student ability, urbanicity, diversity in 
student population, and algebra or pre-algebra tracks. In five sites, there were both CMP and comparison classrooms. 
In the other four CMP sites, comparison classrooms were recruited from other locations. Data for the sixth- and 
seventh-grade analytic samples were collected in the 1994-95 school year and included 36 CMP classrooms and 18 
comparison classrooms. Data for the eighth-grade sample was collected in the 1995-96 school year, and included 14 
CMP classrooms and seven comparison classrooms. The WWC based its effectiveness rating on outcomes com- 
bined across students in all three grades. Although some intervention students used CMP in a previous school year, 
the findings from this study measure the effectiveness of receiving 1 year of the intervention because the pre-interven- 
tion measures were assessed at the beginning of same school year in which outcomes were measured. The analytic 
sample included a total of 2,456 students across these 75 classrooms. The study did not specify which edition of 
CMP was used, but based on the timing of the study, it was likely the first edition. 


Table 2. Scope of reviewed research 


Grade 

6,7,8 

Delivery method 

Whole class 

Intervention type 

Curriculum 
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Effectiveness Summary 

The WWC review of CMP for the Primary Mathematics topic area includes student outcomes in one domain: math- 
ematics achievement. The findings below present the authors’ estimates and WWC-calculated estimates of the 
size and statistical significance of the effects of CMP on primary students. Additional comparisons are presented 
as supplemental findings in Appendix D. These supplemental findings do not factor into the intervention’s rating of 
effectiveness. For a more detailed description of the rating of effectiveness and extent of evidence criteria, see the 
WWC Rating Criteria on p. 23. 


Summary of effectiveness for the comprehension domain 

Table 3. Rating of effectiveness and extent of evidence for the mathematics achievement domain 


Rating of effectiveness 

Criteria met 

No discernible effects 

No evidence of statistically 
significant or substantively 
important effects, either positive 
or negative. 

In the two studies that reported findings, the estimated impact of the intervention on outcomes in the 
mathematics achievement domain was neither statistically significant nor large enough to be considered 
substantively important. 

Extent of evidence 

Criteria met 

Medium to large 

Two studies that included 3,062 students across at least 23 schools reported evidence of effectiveness in the 
mathematics achievement domain. 


Two studies that meet WWC group design standards with reservations reported findings in the mathematics 
achievement domain. 

Cai et al. (2011) reported, and the WWC confirmed (after applying a correction for classroom-level clustering), no 
statistically significant or substantively important difference between the CMP group and the comparison group in 
the mathematics achievement domain. The WWC characterizes these study findings as an indeterminate effect. 

Ridgway et al. (2002) reported a statistically significant difference between the CMP group and the comparison 
group in the mathematics achievement domain. However, after applying a correction for classroom-level cluster- 
ing, the WWC found that this difference was no longer statistically significant and the result was not substantively 
important. The WWC characterizes these study findings as an indeterminate effect. 

Thus, for the mathematics achievement domain, neither study showed effects that were statistically significant 
nor large enough to be considered substantively important. This results in a rating of no discernible effects, with a 
medium to large extent of evidence. 
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24(3), 367. This study is ineligible for review because it does not use an eligible design. 

Moyer, J. C., Cai, J., Wang, N., & Nie, B. (2011). Impact of curriculum reform: Evidence of change in classroom 
practice in the United States. International Journal of Educational Research, 50(2), 87-99. This study is ineli- 
gible for review because it is out of scope of the protocol. 

Newton, J., Geller, R., Umbeck, L., & Kasmer, L. (2012). Reflections on teaching with a standards-based curriculum: 
A conversation among mathematics educators. Montana Mathematics Enthusiast, 9(1), 179-192. This study is 
ineligible for review because it does not use an eligible design. 

Newton, J. A. (2012). Investigating the mathematical equivalence of written and enacted middle school standards- 
based curricula: Focus on rational numbers. International Journal of Educational Research, 51 , 66-85. This 
study is ineligible for review because it does not use an eligible design. 
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cation, 45(5), 699-71 1 . This study is ineligible for review because it is out of scope of the protocol. 
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Appendix A.1: Research details for Cai et al. (2011) 

Cai, J., Wang, N., Moyer, J. C., Wang, C., & Nie, B. (2011). Longitudinal investigation of the curricular 
effect: An analysis of student learning outcomes from the LieCal project in the United States. 
International Journal of Educational Research, 50(2), 1 1 7-1 36. 10 

Additional sources: 

Cai, J. (2015). Curriculum reform and mathematics learning: Evidence from two longitudinal studies. 
In S. J. Cho (Ed.), Selected regular lectures from the 12th International Congress on Mathemati- 
cal Education (pp. 71-92). Gewerbestrasse, Switzerland: Springer International Publishing. 

Cai, J., Moyer, J. C., & Wang, N. (2013). Longitudinal investigation of the effect of middle school cur- 
riculum on learning in high school. In A. Lindmeier & A. Heinze (Eds.), The proceedings of the 
37th conference of the International Group for the Psychology of Mathematics Education (pp. 
137-144). Kiel, Germany: The International Group for the Psychology of Mathematics Education. 

Cai, J., Moyer, J. C., Wang, N., Hwang, S., Nie, B., & Garber, T. (2013). Mathematical problem posing as a 
measure of curricular effect on students’ learning. Educational Studies in Mathematics, 83(1), 57-69. 


Table Al. Summary of findings 


Meets WWC group design standards with reservations 

Outcome domain 

Sample size 

Study findings 

Average improvement index 

(percentile points) Statistically significant 

Mathematics achievement 

14 schools/606 students 

+4 No 


Setting The study was conducted in 14 middle schools, all of which were located in a single large, 
urban school district in the United States. 

Study sample Seven middle schools that implemented CMP in grades 6-8 were selected for the study. Another 
seven middle schools in the district that were not implementing CMP were selected for the study’s 
comparison group, based on their similarity to the CMP schools on demographic characteristics. 
The study sample consisted of students who began sixth grade in fall 2005. Students in both the 
CMP and comparison groups were assessed in fall 2005 (at the beginning of sixth grade), and in 
spring 2006, spring 2007, and spring 2008 (at the end of each grade). The eighth-grade analytic 
sample, assessed in spring 2008, consisted of 606 students with an equal number of students and 
schools in the intervention and comparison groups. The analytic sample was about half the size 
of the baseline sample, which included 1 ,284 students. About 85% of the students in the baseline 
sample were minorities: 64% were African American, 16% were Hispanic, 4% were Asian, and 1 % 
were Native American. The remaining 1 5% of the students were White. 

In addition, the CMP and comparison students were tracked into high school and outcomes 
were assessed at the end of each grade in ninth, tenth, and eleventh grades (spring 2009, spring 
2010, and spring 2011). In the 10 high schools that were included in this follow-up sample, the 
CMP and comparison students were mixed together in the same classrooms and used the same 
(non-CMP) curricula. 

The eighth-grade findings are considered the main outcomes in this review and presented in 
Appendix C because they are the most immediate outcome measuring the 3 full years of CMP 
use. The sixth-, seventh-, and eleventh-grade outcomes are considered supplemental findings 
presented in Appendix D that do not factor into the intervention’s rating of effectiveness. 11 
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Intervention 

group 


Comparison 

group 


Outcomes and 
measurement 


Support for 
implementation 


Students in the intervention schools used the first edition of CMP (version 1) as their core 
mathematics curriculum in the sixth, seventh, and eighth grades in the 2005-06 through 
2007-08 school years; specific details about how CMP was implemented in study schools are 
not provided by the authors. 

Students in the comparison schools used one of several traditional mathematics curricula 
already in use in their schools during each grade (grades 6, 7, and 8); specific details about 
how the comparison curricula were implemented are not provided by the authors. The authors 
conducted detailed analyses on the curricular materials to examine differences between CMP 
and one of the curricula used by the comparison group (Glencoe). The authors noted that 
there were differences between CMP and non -CMP curriculum; notably, CMP emphasizes 
problem solving while the non-CMP curricula take a more traditional approach that focuses on 
concepts and procedures. The authors also indicate that there were some differences between 
the different non-CMP curricula, but the differences between the comparison curricula were 
not substantial since they took the same traditional approach to math instruction. The authors 
did not name the other math curricula used by students in the comparison group. 

The study included two outcome measures that meet WWC review requirements and fall 
within the mathematics achievement domain: (a) open-ended tasks and (b) ability to pose 
problems. Both assessments were developed by the researchers involved in the study. The 
first assessment was administered four times: at the beginning of sixth grade (fall 2005), at the 
end of sixth grade (spring 2006), at the end of seventh grade (spring 2007), and at the end of 
eighth grade (spring 2008). The second outcome was measured at the end of eleventh grade 
(spring 201 1). For a more detailed description of the outcome measures, see Appendix B. 

The study also examined five outcomes that do not meet WWC standards. Three middle 
school assessments were used in analyses that did not meet standards because the CMP and 
comparison groups were not found to be equivalent at baseline: (a) translation, (b) computa- 
tion, and (c) equation solving. Two additional outcomes, the Classroom Assessment Based 
on Standards (CABS) administered in ninth grade and the state mathematics achievement 
test administered in tenth grade, do not meet standards because the authors did not provide 
evidence of baseline equivalence for the analytic sample. 

Five other outcomes used by the authors were not eligible for review. Three middle school 
outcomes are ineligible because they are measures of implementation fidelity: (a) the level of 
conceptual and procedural emphasis in lessons, (b) the difficulty of instructional tasks, and (c) 
the difficulty of homework problems. Two eleventh-grade outcomes are ineligible because they 
were used in an analysis that drew on an ineligible design (posttest only): a graphing task and 
equation solving task. 

The authors did not provide any information on support for implementation. 
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Appendix A.2: Research details for Ridgway et al. (2002) 

Ridgway, J. E., Zawojewski, J. S., Hoover, M. N., & Lambdin, D. V. (2002). Student attainment in the Con- 
nected Mathematics curriculum. In S. L. Senk & D. R. Thompson (Eds.), Standards-based school 
mathematics curricula: What are they? What do students learn? (pp. 193-224). Mahwah, NJ: Law- 
rence Erlbaum Associates, Inc. 12 

Table A2. Summary of findings Meets WWC group design standards with reservations 


Study findings 
Average improvement index 

Outcome domain Sample size (percentile points) Statistically significant 


Mathematics achievement 9 sites/2,456 students 0 No 


Setting 

The study was conducted in nine sites across the United States (five in the Midwest, two in 


the West, and two in the East). The authors do not indicate whether a site is a single school or 
school district. 

Study sample 

The study sample consisted of sixth- and seventh-grade students in the 1994-95 school year 
and eighth-grade students in the 1995-96 school year. The intervention and comparison group 
participants were matched to the extent possible on ability, location, and diversity in student 
population. In five of the nine sites that participated in the study, only a small number of teach- 
ers were using CMP , so comparison classrooms were selected locally. At the four other sites, 
comparison classrooms were identified in alternate locations. At each site, pairs of classrooms 
were selected within each grade level to form the intervention group; one comparison class- 
room was selected for every pair of intervention classrooms. The 1994-95 sample included 

338 sixth-grade students and 627 seventh-grade students from 36 classrooms (18 in each 
grade) who used the CMP curriculum and 162 sixth-grade students and 234 seventh-grade 
students from 18 comparison group classrooms (nine in each grade). The 1995-96 sample 
included 820 eighth-grade students from 14 classrooms using CMP and 275 students from 
seven comparison classrooms. The authors provided results by grade. 

For this review, student data were combined across grades; the effectiveness rating is based 
on the combined analyses for each outcome measure and presented in Appendix C. 13 Although 
some intervention students in this combined analysis used CMP in a previous school year, the 
combined finding measures the effectiveness of receiving 1 year of the intervention because the 
pre-intervention measures were assessed at the beginning of same school year in which out- 
comes were measured. The authors did not report demographic characteristics of the 
study students. 

Intervention 

group 

Students in the intervention group used CMP as their core math curriculum. Specific details 
about how CMP was implemented in study schools are not provided by the authors. The 
sixth- and seventh-grade intervention students used CMP in the 1994-95 school year, and 
the eighth-grade intervention students used CMP in the 1995-96 school year. The sixth-grade 
students had no prior use of CMP ; however, approximately three-fourths of the seventh- and 
eighth-grade students had used CMP in the previous year. The authors did not indicate which 
edition of CMP was used, but it was likely the first edition of CMP, since the study was con- 
ducted between 1 994-96 and the second edition of CMP was not developed until 2000. 
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Comparison 

group 


Outcomes and 
measurement 


Support for 
implementation 


Students in the comparison group used commercially available mathematics textbooks. The 
authors did not provide the name of the comparison texts, nor did they provide details about 
how the comparison curricula were implemented in study schools. Teachers in the comparison 
group did not use the CMP curriculum and implemented their regular curriculum. 

The study included two outcome measures, both of which meet the review requirements and 
fall within the math achievement domain: (a) the Iowa Test of Basic Skills (ITBS, a standard- 
ized test) and (b) the Balanced Assessment (BA) test (developed as a collaboration between 
the study authors and the Balanced Assessment Project). Both tests were administered to the 
study students in the fall and spring of the 1994-95 school year for sixth- and seventh-grade 
students and in the fall and spring of the 1995-96 school year for eighth-grade students. 

The grade-level analyses for the ITBS and eighth-grade analyses for the BA do not demon- 
strate equivalence of the analytic intervention and comparison groups. Only the sixth- and 
seventh-grade-level analyses for the BA test demonstrate equivalence. These BA outcomes 
are presented as supplemental findings in Appendix D. For a more detailed description of both 
outcome measures, see Appendix B. 

All CMP teachers attended a summer CMP workshop at Michigan State University. This work- 
shop included sessions that involved teachers experiencing the curriculum as students as well 
as sessions to share methods and techniques for implementation. The authors indicate that 
they do not have information on how CMP materials were used in the classroom. 
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Appendix B: Outcome measures for the mathematics achievement domain 


Mathematics achievement 

Balanced Assessment (BA) test 

The BA test was developed in collaboration between the Balanced Assessment Project and study authors to 
assess reasoning, mathematical communication, mathematical problem solving, and the ability to make connec- 
tions among mathematical concepts. The BA test does not mimic the language or content of the CMP curricu- 
lum and is therefore not considered overaligned with the intervention. The BA instrument assessed a variety of 
curricular topics and consisted of constructed-response items that required a range of responses: items required 
students to interpret real-world situations, explain their strategies, and provide justifications for their responses. 
Each item varied in the number of points awarded for a correct answer. Five different forms of the BA test were 
used. BA tests were scored by trained study staff (which included mathematics teachers, mathematics educa- 
tion graduate students, senior high school students, and professionals who attended a structured training) using 
rubrics and guidelines under the direction of the CMP and Balanced Assessment Project staff. The BA tests had 
a reported inter-rater reliability of at least 90% (as cited in Ridgway et al., 2002). This outcome is only reported 
as a supplemental finding in Appendix D. 

Iowa Test of Basic Skills (ITBS) 

The ITBS includes five subtests (Concepts, Estimation, Problem Solving, Data Interpretation, and Computation) 
which collectively assess numbers and operations skills and concepts, as in other standardized tests. The test 
includes 60 timed multiple-choice items, each worth one point. The ITBS tests were scored by the Riverside 
Testing Company (as cited in Ridgway et al., 2002). 

Open-ended tasks 

This test included students’ scores on five open-ended tasks (combined into one score) adapted from the 

BA. The tasks tested students' high-level thinking skills as well as procedural knowledge and reasoning in 
problem-solving. Two middle school math teachers were trained to code and score the tests. The exact agree- 
ment between two coders was approximately 80% and agreement within one point was over 95%. The authors 
reported internal consistency (Cronbach's Alpha) for each measurement occasion, ranging from .65 to .77 
across forms and measurement occasions (as cited in Cai et al., 2011). 

Problem-posing ability 

Problem-posing ability was a researcher-developed task that involved identifying an equation for a given graph 
and posing a real-life situation that could be represented by the graph. Students’ answers were scored by two 
researchers using a rubric. Agreement between coders ranged from 92% to 100% (as cited in Cai et al., 2011). 
This outcome is only reported as a supplemental finding in Appendix D. 
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Appendix C: Findings included in the rating for the mathematics achievement domain 





Mean 

(standard deviation) 

WWC calculations 


Outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Cai etal. (2011) a 

Open-ended tasks total score 

Grade 8 

14 schools/ 
606 students 

575.00 

(95.00) 

565.00 

(100.00) 

10.0 

0.10 

+4 

>.05 

Domain average for mathematics achievemtn (Cai et al., 2011) 



0.10 

+4 

Not 

statistically 

significant 

Ridgway et al. (2002) b 

Balanced Assessment (BA) 

Grades 

6,7 

9 sites/ 

1,361 students 

nr 

nr 

na 

na 

na 

Grade 6 < .001 
Grade 7 < .001 

Iowa Test of Basic Skills (ITBS) 

Grades 

6,7,8 

9 sites/ 

2,456 students 

8.78 

(2.88) 

8.77 

(2.97) 

0.01 

0.00 

0 

Grade 6 < .001 
Grade 7 < .001 
Grade 8 .053 

Domain average for mathematics achievement (Ridgway et al., 2002) 



0.00 

0 

Not 

significantly 

significant 

Domain average for mathematics achievement across all studies 



0.05 

+2 

na 


Table Notes: For mean difference, effect size, and improvement index values reported in the table, a positive number favors the intervention group and a negative number favors 
the comparison group. The effect size is a standardized measure of the effect of an intervention on outcomes, representing the average change expected for all individuals who are 
given the intervention (measured in standard deviations of the outcome measure). The improvement index is an alternate presentation of the effect size, reflecting the change in 
an average individual’s percentile rank that can be expected if the individual is given the intervention. The WWC-computed average effect size is a simple average rounded to two 
decimal places; the average improvement index is calculated from the average effect size. The statistical significance of the study’s domain average was determined by the WWC. 
Some statistics may not sum as expected due to rounding, na = not available, nr = not reported. 

a For Cai et al. (2011), means and standard deviations in the table were obtained through an author query. A correction for clustering was needed but did not affect whether any of the 
contrasts were found to be statistically significant. The p-value presented here was reported in the original study. The WWC calculated the intervention group mean using a difference- 
in-differences approach by adding the impact of the intervention (i.e., difference in mean gains between the intervention and comparison groups) to the unadjusted comparison group 
posttest means. This study is characterized as having an indeterminate effect because the estimated effect is neither statistically significant nor substantively important. For more 
information, please refer to the WWC Standards and Procedures Handbook (version 3.0), p. 26. 

b For Ridgway et al. (2002), the authors reported unadjusted ITBS means and standard deviations separately by grades 6, 7, and 8; the grades were pooled together by the WWC. 

The BA did not demonstrate equivalence when pooled across grades 6, 7, and 8, nor did it demonstrate equivalence for the grade 8 subgroup; therefore, the WWC pooled together 
grades 6 and 7. For the BA, the authors reported p-values using the results from an ANCOVA model, but did not report information needed to calculate a WWC effect size. The p-values 
presented here were reported in the original study. Corrections for clustering and multiple comparisons were needed, but the authors did not provide enough information to determine 
WWC significance for the BA; the WWC-computed p-value is .97 for the ITBS; therefore, the WWC does not find the result to be statistically significant. The WWC calculated the ITBS 
intervention group mean using a difference-in-differences approach by adding the impact of the intervention (i.e., difference in mean gains between the intervention and comparison 
groups) to the unadjusted comparison group posttest means. The WWC excludes findings without an effect size from the domain averages. This study is characterized as having an 
indeterminate effect because the estimated effect is neither statistically significant nor substantively important. For more information, please refer to the WWC Standards and Proce- 
dures Handbook (version 3.0), p. 26. 
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Appendix D: Description of supplemental findings for the mathematics achievement domain 


Mean 

(standard deviation) WWC calculations 


Outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Cai etal. (2011) a 

Open-ended tasks total score 

Grade 6 

14 schools/ 
606 students 

494.00 

(97.00) 

502.00 

(97.00) 

-8.00 

-0.08 

-3 

nr 

Open-ended tasks total score 

Grade 7 

14 schools/ 
606 students 

538.00 

(92.00) 

531.00 

(93.00) 

7.00 

0.08 

+3 

nr 

Problem-posing performance 

Grade 11 

(3-year follow-up) 

136 students 

1.56 

(1.14) 

0.50 

(1.17) 

1.06 

0.11 

+4 

nr 

Problem-posing performance 

Grade 11, middle 
third subgroup 
(3-year follow-up) 

45 students 

0.27 

(0.87) 

0.05 

(0.23) 

0.22 

0.32 

+12 

nr 

Problem-posing performance 

Grade 11, bottom 
third subgroup 
(3-year follow-up) 

45 students 

0.11 

(0.72) 

0.00 

(0.00) 

0.11 

0.48 

+18 

nr 

Ridgway et al. (2002) b 

Balanced Assessment test 

Grade 6 

27 classrooms/ 
500 students 

nr 

nr 

na 

na 

na 

<.001 

Balanced Assessment test 

Grade 7 

27 classrooms/ 
861 students 

nr 

nr 

na 

na 

na 

< .001 


Table Notes: The supplemental findings presented in this table are additional findings from studies in this report that meet WWC design standards with or without reservations, 
but do not factor into the determination of the intervention rating. For mean difference, effect size, and improvement index values reported in the table, a positive number favors 
the intervention group and a negative number favors the comparison group. The effect size is a standardized measure of the effect of an intervention on outcomes, representing 
the average change expected for all individuals who are given the intervention (measured in standard deviations of the outcome measure). The improvement index is an alternate 
presentation of the effect size, reflecting the change in an average individual’s percentile rank that can be expected if the individual is given the intervention. Some statistics may 
not sum as expected due to rounding, na = not available, nr = not reported. 

a For Cai et ai. (201 1 ), the unadjusted means and standard deviations reported in this table were obtained through an author query. The authors did not provide p-values with these 
data. Corrections for multiple comparisons were not implemented because the authors did not report p-values and the WWC-computed p-values did not indicate any of these findings 
were statistically significant. The eighth-grade findings are considered the main outcomes in this review and are presented in Appendix C because they are the most immediate 
outcomes measuring the 3 full years of CMP use. The sixth- and seventh-grade findings (which represent 1 and 2 years of CMP use, respectively) and eleventh-grade outcomes 
(which represent a 3 year follow-up) that are presented in this table are considered supplemental findings. For the eleventh-grade outcomes, the author provided data separately for 
the bottom, middle, and top thirds of students. The WWC examined results separately by the achievement subgroups, and combined data to estimate the effect for the full eleventh- 
grade sample. The author examined different eleventh-grade outcomes using two different baseline tests: an open-ended task and an equation-solving task. The full eleventh-grade 
sample demonstrated baseline equivalence on the open-ended task but not on the equation-solving task. The subgroup analyses for the eleventh grade only met WWC baseline 
equivalence standards on the open-ended task for the middle third achievement level subgroup and the equation-solving test for the bottom third achievement subgroup. The WWC 
calculated the intervention group mean using a difference-in-differences approach by adding the impact of the intervention (i.e., difference in mean gains between the intervention 
and comparison groups) to the unadjusted comparison group posttest means for all outcomes except the eleventh-grade problem-posing performance (bottom third subgroup). 
Please see the WWC Procedures and Standards Handbook (version 3.0) for more information. 

b For Ridgway et al. (2002), the authors reported p-values using the results from an ANCOVA model, but did not report adjusted means and standard deviations. Corrections for cluster- 
ing and multiple comparisons were needed but did not affect whether any of the contrasts were found to be statistically significant. The p-values presented here were reported in the 
original study. 
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Endnotes 

1 The descriptive information for this intervention was obtained from a publicly available source: the intervention’s website (https:// 
connectedmath.msu.edu). The WWC requests developers review the intervention description sections for accuracy from their per- 
spective. The intervention description was provided to the developer in June 2015; however, the WWC received no response. Further 
verification of the accuracy of the descriptive information for this intervention is beyond the scope of this review. 

2 The WWC previously released a report on CMP under the Middle School Mathematics topic area in January 2010; the report was 
prepared using the WWC Procedures and Standards Handbook (version 1 .0) and the Middle School Mathematics review protocol 
(version 1 .0). In June 2015, the WWC restructured the reviews of research on math interventions into two areas instead of three. These 
two review areas are Primary Mathematics (which includes interventions in which math is presented through multi-topic materials and 
curricula, typically used in grades K-8), and Secondary Mathematics (which includes interventions that are organized by math content 
area [e.g., Algebra, Geometry, and Calculus], typically taught in grades 9-12). These two areas replaced the prior Elementary School 
Math, Middle School Math, and High School Math areas, which were organized by student grade level. The WWC is updating and 
replacing intervention reports written under the prior topic areas. 

The literature search for the current report reflects documents publicly available by March 2016. This report has been updated to include 
reviews of 40 studies that were not included in the prior report. Of the additional studies, 31 were not within the scope of the review pro- 
tocol for the Primary Mathematics topic area, and eight were within the scope of the review protocol for the Primary Mathematics topic 
area but did not meet WWC group design standards. One study meets WWC group design standards with reservations, and findings 
from this study are summarized in this report. A complete list and disposition of all studies reviewed are provided in the references. 

The report includes reviews of all studies included in the previous report and resulted in a revised disposition for two studies. Ridgway 
et al. (2002) received a disposition in this report of meets WWC group design standards with reservations, where it had previously 
received the rating of does not meet WWC group design standards. The study received the previous rating because the analytic sam- 
ple included some students that had prior use of the intervention. This review is based on the Primary Mathematics topic area review 
protocol (version 3.1), which does not prohibit prior use of an intervention, as long as the study demonstrates baseline equivalence on 
the analytic sample and meets all other review requirements. Using the WWC Procedures and Standards Handbook (version 3.0) and 
the Primary Mathematics review protocol (version 3.1), several analytic contrasts within this study meet WWC group design standards, 
and therefore, the study is now rated meets WWC group design standards with reservations. 

Schneider (2000) received a disposition in this report of does not meet WWC group design standards, where it had previously received 
a rating of meets WWC group design standards with reservations. This study has baseline differences in the adjustment range (0.05 to 
0.25 standard deviations), and therefore must include an appropriate statistical adjustment. However, an appropriate statistical adjust- 
ment was not conducted by the author. Therefore, the prior rating was incorrect and is corrected in this report. The study is now rated 
does not meet WWC group design standards. 

3 The studies in this report were reviewed using the standards from the WWC Procedures and Standards Handbook (version 3.0), and 
the Primary Mathematics review protocol (version 3.1). The evidence presented in this report is based on available research. Findings 
and conclusions may change as new research becomes available. 

4 This represents the total analytic sample size across the two studies. Cai et al. (201 1) included students from 14 schools within one 
school district. Ridgway et al. (2002) included students from nine sites across the country; the authors did not define whether a site is 
a single school, a school district, or a larger entity. 

5 Please see the Primary Mathematics review protocol (version 3.1) for more information about the outcome domain. 

6 For criteria used in the determination of the rating of effectiveness and extent of evidence, see the WWC Rating Criteria on p. 23. 
These improvement index numbers show the average and range of individual-level improvement indices for all findings across the 
studies. 

7 This review does not include any studies of CMP Algebra I since it is not eligible for review under the WWC’s Primary Mathematics 
review protocol. 

8 One study that meets standards and is summarized in this report provides evidence of effectiveness about the first edition of CMP. 
The other study that meets standards and is summarized in this report may also provide evidence of effectiveness about the first edi- 
tion of CMP. While the author does not specify the edition used, due to the timing of the study, it likely provides evidence about the 
first edition of CMP. 

9 For the analyses of eleventh-grade outcomes, the authors followed students who enrolled in the 10 high schools with the largest 
number of CMP and non-CMP students from the original sample. The eleventh-grade outcomes included 135-136 students from the 
original sample of nearly 700 students. The number of classes from which the eleventh-grade sample originated was not reported by 
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the author. In high school, the CMP and non-CMP students were no longer separated and they used the same curriculum (which was 
not CMP). 

10 The WWC identified three additional sources related to Cai et al. (201 1). These studies do not contribute unique information to 
Appendix A.1 and are not listed here. 

11 The ninth- and tenth-grade analytic samples did not demonstrate equivalence as required, and therefore do not meet WWC group 
design standards; thus, they are not presented in Appendix D. 

12 The WWC identified one additional source related to Ridgway et al. (2002). The study does not contribute unique information to 
Appendix A.2 and is not listed here. 

13 Although the study presented findings for each grade separately, the WWC combined this data when presenting the findings in 
Appendix C. Within each study and outcome domain, the WWC reports a set of primary findings from multiple outcomes that use 
comparable samples, when possible. When the WWC combined the data across all three grades for each outcome measure, baseline 
equivalence was demonstrated on the ITBS, but not on the BA. Therefore, we report as the primary findings the combined grades 6 
and 7 sample for the BA, and the combined sample across all three grades for the ITBS. In addition, the two grade subgroups that 
demonstrated equivalence on the BA are presented as supplementary results in Appendix D. 

Recommended Citation 

U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2017, February). 

Primary Mathematics intervention report: Connected Mathematics Project. Retrieved from https://whatworks.ed.gov 
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WWC Rating Criteria 

Criteria used to determine the rating of a study 

Study rating 

Criteria 

Meets WWC group design 
standards without reservations 

A study that provides strong evidence for an intervention’s effectiveness, such as a well-implemented RCT. 

Meets WWC group design 

A study that provides weaker evidence for an intervention's effectiveness, such as a QED or an RCT with high 

standards with reservations 

attrition that has established equivalence of the analytic samples. 

Criteria used to determine the rating of effectiveness for an intervention 

Rating of effectiveness 

Criteria 

Positive effects 

Two or more studies show statistically significant positive effects, at least one of which met WWC group design 
standards for a strong design, AND 

No studies show statistically significant or substantively important negative effects. 

Potentially positive effects 

At least one study shows a statistically significant or substantively important positive effect, AND 

No studies show a statistically significant or substantively important negative effect AND fewer or the same number 
of studies show indeterminate effects than show statistically significant or substantively important positive effects. 

Mixed effects 

At least one study shows a statistically significant or substantively important positive effect AND at least one study 
shows a statistically significant or substantively important negative effect, but no more such studies than the number 
showing a statistically significant or substantively important positive effect, OR 

At least one study shows a statistically significant or substantively important effect AND more studies show an 
indeterminate effect than show a statistically significant or substantively important effect. 

Potentially negative effects 

One study shows a statistically significant or substantively important negative effect and no studies show 
a statistically significant or substantively important positive effect, OR 

Two or more studies show statistically significant or substantively important negative effects, at least one study 
shows a statistically significant or substantively important positive effect, and more studies show statistically 
significant or substantively important negative effects than show statistically significant or substantively important 
positive effects. 

Negative effects 

Two or more studies show statistically significant negative effects, at least one of which met WWC group design 
standards for a strong design, AND 

No studies show statistically significant or substantively important positive effects. 

No discernible effects 

None of the studies shows a statistically significant or substantively important effect, either positive or negative. 

Criteria used to determine the extent of evidence for an intervention 

Extent of evidence 

Criteria 

Medium to large 

The domain includes more than one study, AND 

The domain includes more than one school, AND 

The domain findings are based on a total sample size of at least 350 students, OR, assuming 25 students in a class, 
a total of at least 14 classrooms across studies. 

Small 

The domain includes only one study, OR 

The domain includes only one school, OR 

The domain findings are based on a total sample size of fewer than 350 students, AND, assuming 25 students 
in a class, a total of fewer than 14 classrooms across studies. 
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Glossary of Terms 

Attrition 

Clustering adjustment 
Confounding factor 

Design 
Domain 
Effect size 

Eligibility 

Equivalence 

Extent of evidence 

Improvement index 

Intervention 
Intervention report 


Multiple comparison 
adjustment 

Quasi-experimental 
design (QED) 

Randomized controlled 
trial (RCT) 

Rating of effectiveness 


Single-case design 


Attrition occurs when an outcome variable is not available for all participants initially assigned 
to the intervention and comparison groups. The WWC considers the total attrition rate and 
the difference in attrition rates across groups within a study. 

If intervention assignment is made at a cluster level and the analysis is conducted at the student 
level, the WWC will adjust the statistical significance to account for this mismatch, if necessary. 

A confounding factor is a component of a study that is completely aligned with one of the 
study conditions, making it impossible to separate how much of the observed effect was 
due to the intervention and how much was due to the factor. 

The design of a study is the method by which intervention and comparison groups were assigned. 
A domain is a group of closely related outcomes. 

The effect size is a measure of the magnitude of an effect. The WWC uses a standardized 
measure to facilitate comparisons across studies and outcomes. 

A study is eligible for review and inclusion in this report if it falls within the scope of the 
review protocol and uses either an experimental or matched comparison group design. 

A demonstration that the analysis sample groups are similar on observed characteristics 
defined in the review area protocol. 

An indication of how much evidence supports the findings. The criteria for the extent 
of evidence levels are given in the WWC Rating Criteria on p. 23. 

Along a percentile distribution of individuals, the improvement index represents the gain 
or loss of the average individual due to the intervention. As the average individual starts at 
the 50th percentile, the measure ranges from -50 to +50. 

An educational program, product, practice, or policy aimed at improving student outcomes. 

A summary of the findings of the highest-quality research on a given program, product, 
practice, or policy in education. The WWC searches for all research studies on an interven- 
tion, reviews each against design standards, and summarizes the findings of those that 
meet WWC design standards. 

When a study includes multiple outcomes or comparison groups, the WWC will adjust 
the statistical significance to account for the multiple comparisons, if necessary. 

A quasi-experimental design (QED) is a research design in which study participants are 
assigned to intervention and comparison groups through a process that is not random. 

A randomized controlled trial (RCT) is an experiment in which eligible study participants are 
randomly assigned to intervention and comparison groups. 

The WWC rates the effects of an intervention in each domain based on the quality of the 
research design and the magnitude, statistical significance, and consistency in findings. The 
criteria for the ratings of effectiveness are given in the WWC Rating Criteria on p. 23. 

A research approach in which an outcome variable is measured repeatedly within and 
across different conditions that are defined by the presence or absence of an intervention. 
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Glossary of Terms 


Standard deviation The standard deviation of a measure shows how much variation exists across observations 
in the sample. A low standard deviation indicates that the observations in the sample tend 
to be very close to the mean; a high standard deviation indicates that the observations in 
the sample tend to be spread out over a large range of values. 

Statistical significance Statistical significance is the probability that the difference between groups is a result of 

chance rather than a real difference between the groups. The WWC labels a finding statistically 
significant if the likelihood that the difference is due to chance is less than 5% (p < .05). 


Substantively important a substantively important finding is one that has an effect size of 0.25 or greater, regardless 

of statistical significance. 

Systematic review a review of existing literature on a topic that is identified and reviewed using explicit meth- 
ods. A WWC systematic review has five steps: 1) developing a review protocol; 2) searching 
the literature; 3) reviewing studies, including screening studies for eligibility, reviewing the 
methodological quality of each study, and reporting on high quality studies and their find- 
ings; 4) combining findings within and across studies; and, 5) summarizing the review. 


Please see the WWC Procedures and Standards Handbook (version 3.0) for additional details. 


Connected Mathematics Project Updated February 2017 


Page 25 


WWC Intervention Report 



An intervention report summarizes the findings of high-quality research on a given program, practice, or policy in 
education. The WWC searches for all research studies on an intervention, reviews each against evidence standards, 
and summarizes the findings of those that meet standards. 


This intervention report was prepared for the WWC by Mathematica Policy Research under contract ED-IES-13-C-0010. 
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